Technical Field
The present disclosure relates to generally to image registration or image matching. More particularly, the present disclosure relates to phase correlation for image registration. Specifically, the present disclosure relates to an image registration system having an enhanced phase correlation system and process for image matching.
Background Information
Generally, image registration is the process of transforming different sets of data into one coordinate system, and aligning the transformed data. The data may be multiple photographs, data from different sensors, times, depths, or viewpoints. It is used in computer vision, medical imaging, biological imaging and brain mapping, military automatic target recognition, and compiling and analyzing images and data from satellites. Image registration is required to compare or integrate the data obtained from these different measurements.
Registering or matching images that have fundamentally different characteristics is a difficult task. Exemplary fundamentally different characteristics include but are not limited to underlying phenomenology, temporal differences, or matching predictions to real imagery. Feature-based methods, which are the registration (matching) methods of choice for easier applications, may not work well when the images or data contain sparse features or features that are not salient across image modalities.
For geo-registration of difficult imagery, conventional registration methods may not be able to automatically register frames of video to reference imagery due to appearance differences. In another example, when registering synthetic aperture radar (SAR) imagery to electro-optical (EO) reference imagery, the drastic differences in phenomenology may prevent most automatic registration algorithms from succeeding. Stated otherwise, conventional registration algorithms have significant difficulty registering SAR imagery to EO imagery. These difficult image registration scenarios may defeat traditional normalized correlation methods, phase correlation methods, mutual information methods, and feature-based matching methods.
Generally, spatial image registration methods operate in the image domain and match intensity patterns or features in images. Some of the feature matching algorithms are outgrowths of traditional techniques for performing manual image registration, in which an operator chooses corresponding control points (CP) in images, such as a cross or steeple on a church or a geographic reference such as a rocky outcropping. When the number of control points exceeds the minimum required to define the appropriate transformation model, iterative algorithms like random sample consensus (RANSAC) can be used to estimate the parameters of a particular transformation type (e.g. affine) for registration of the images.
Generally, frequency-domain image registration methods find the transformation parameters for registration of the images while working in the frequency domain. Note, a given function or signal can be converted between the time and frequency domains with a pair of mathematical operators called a transform. An example is the Fourier transform, which converts the time function into a sum of sine waves of different frequencies, each of which represents a frequency component.
The ‘spectrum’ of frequency components is the frequency domain representation of the signal. Such methods work for detecting simple transformations, such as translation, rotation, and scaling. Applying a phase correlation method to a pair of images (i.e., the test image and the reference image) may produce a third image which contains a single peak. The location of this peak corresponds to the relative translation between the images. Unlike many spatial-domain algorithms, the phase correlation method is resilient to noise, occlusions, and other defects typical of medical or satellite images. Additionally, phase correlation method uses the fast Fourier transform to compute the cross-correlation between the test image and the reference image, generally resulting in large performance gains. The method can be extended to determine rotation and scaling differences between two images by first converting the images to log-polar coordinates. Due to properties of the Fourier transform, the rotation and scaling parameters can be determined in a manner invariant to translation.
Background—Phase Correlation Method
Phase correlation works by exploiting the shift property of Fourier transforms to map spatial domain translations to frequency domain linear functions. Let IT(x, y), IR(x, y) represent the test image and reference image, respectively, and assume the test image (IT) is a translation of the reference image (IR):IT(x,y)=IR(x+Δx,y+Δy)  (Equation 1)
Denote the Fourier transform operator by ℑ. According to the Fourier shift theorem, then:ℑ{IT}(ωx,ωy)=ei(ωxΔx+ωyΔy)ℑ{IR}(ωx,ωy)  (Equation 2)
The complex exponential factor in Equation (2) represents a two-dimensional linear phase function. In the spatial domain, the two-dimensional linear phase function corresponds to a delta function since:ℑ−1{ei(ωxΔx+ωyΔy}=δ(x+Δx,y+Δy).  (Equation 3)
The phase correlation approach correlates pre-whitened versions of the test image (IT) and reference image (IR). In the frequency domain, the correlation output (denoted CΦ) takes the following form:
                                          C            Φ                    ⁢                      {                                          I                T                            ,                              I                R                                      }                    ⁢                      (                                          ω                x                            ,                              ω                y                                      )                          =                                            𝔍              ⁢                              {                                  I                  T                                }                            ⁢                              (                                                      ω                    x                                    ,                                      ω                    y                                                  )                            ⁢                              𝔍                *                            ⁢                              {                                  I                  R                                }                            ⁢                              (                                                      ω                    x                                    ,                                      ω                    y                                                  )                                                                                                      𝔍                  ⁢                                      {                                          I                      T                                        }                                    ⁢                                      (                                                                  ω                        x                                            ,                                              ω                        y                                                              )                                                                              ⁢                                                                                    𝔍                    *                                    ⁢                                      {                                          I                      R                                        }                                    ⁢                                      (                                                                  ω                        x                                            ,                                              ω                        y                                                              )                                                                                                .                                    (                  Equation          ⁢                                          ⁢          4                )            
If the test image (IT) is a translated version of the reference image (IR) as in Equation (1), then substituting Equation (2) into Equation (4) yields:
                                          C            Φ                    ⁢                      {                                          I                T                            ,                              I                R                                      }                    ⁢                      (                                          ω                x                            ,                              ω                y                                      )                          =                                                            e                                  i                  ⁡                                      (                                                                                            ω                          x                                                ⁢                        Δ                        ⁢                                                                                                  ⁢                        x                                            +                                                                        ω                          y                                                ⁢                        Δ                        ⁢                                                                                                  ⁢                        y                                                              )                                                              ⁢              𝔍              ⁢                              {                                  I                  R                                }                            ⁢                              (                                                      ω                    x                                    ,                                      ω                    y                                                  )                            ⁢                              𝔍                *                            ⁢                              {                                  I                  R                                }                            ⁢                              (                                                      ω                    x                                    ,                                      ω                    y                                                  )                                                                                                      𝔍                  ⁢                                      {                                          I                      R                                        }                                    ⁢                                      (                                                                  ω                        x                                            ,                                              ω                        y                                                              )                                                                              ⁢                                                                                    𝔍                    *                                    ⁢                                      {                                          I                      R                                        }                                    ⁢                                      (                                                                  ω                        x                                            ,                                              ω                        y                                                              )                                                                                                =                      e                          i              ⁡                              (                                                                            ω                      x                                        ⁢                    Δ                    ⁢                                                                                  ⁢                    x                                    +                                                            ω                      y                                        ⁢                    Δ                    ⁢                                                                                  ⁢                    y                                                  )                                                                        (                  Equation          ⁢                                          ⁢          5                )            and thusℑ−1{C101{IT,IR}(ωx,ωy)}=δ(x+Δx,y+Δy).  (Equation 6)
According to Equation (6), the correlation corresponds to an impulse function in the spatial domain.
In practice, the spatial domain peak in Equation (6) may be spread over multiple pixels, due to noise effects, slight non-translational alignment errors between the test image (IT) and reference image (IR), and non-integer translational offsets. Because of the duality between spatial and frequency domains, an alternative approach to translation estimation is to use Equation (5) to determine the two-dimensional linear phase function in the frequency domain.
Linear regression analysis has been used in an attempt to fit a plane to the two-dimensional linear phase function. However, use of linear regression can be problematic due to problems with surface fitting noisy data and phase wrapping issues.